{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Learning regression with ChatGPT free version"
      ],
      "metadata": {
        "id": "HNwIYLl62qlX"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sure, let's proceed step by step. First, we need to mount Google Drive in our Jupyter Notebook environment so we can access the dataset.\n",
        "\n",
        "Here's how you can do it:\n",
        "\n",
        "```python\n",
        "from google.colab import drive\n",
        "\n",
        "# Mount Google Drive\n",
        "drive.mount('/content/drive')\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `from google.colab import drive`: This line imports the `drive` module from the `google.colab` package.\n",
        "- `drive.mount('/content/drive')`: This command mounts your Google Drive at the specified path `/content/drive`. When you run this cell, it will prompt you to authenticate and give you a link to follow to get an authorization code. Once you enter the code, your Google Drive will be mounted.\n",
        "\n",
        "Let me know if you have any questions before we proceed."
      ],
      "metadata": {
        "id": "X1R2ztb0ymEA"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l-oBO3usybAU",
        "outputId": "a6675fba-0d1c-4b4e-e179-5fc70d040dba"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "# Mount Google Drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! Now that we have mounted Google Drive, we can load the dataset into our notebook.\n",
        "\n",
        "Here's how you can load the dataset:\n",
        "\n",
        "```python\n",
        "import pandas as pd\n",
        "\n",
        "# Define the file path\n",
        "file_path = '/content/drive/My Drive/Chapter05/Ecommerce Customers.csv'\n",
        "\n",
        "# Load the dataset into a DataFrame\n",
        "ecommerce_data = pd.read_csv(file_path)\n",
        "\n",
        "# Display the first few rows of the DataFrame\n",
        "print(ecommerce_data.head())\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `import pandas as pd`: This line imports the pandas library and aliases it as `pd`, which is a common convention.\n",
        "- `file_path = '/content/drive/My Drive/Chapter05/Ecommerce Customers.csv'`: This line defines the file path where the dataset is located in your Google Drive.\n",
        "- `pd.read_csv(file_path)`: This function reads the CSV file located at the specified path and creates a DataFrame from it.\n",
        "- `print(ecommerce_data.head())`: This line prints the first few rows of the DataFrame to verify that the dataset has been loaded correctly.\n",
        "\n",
        "Let me know if you have any questions or if you'd like to proceed to the next step!"
      ],
      "metadata": {
        "id": "9MfHvsTFznzA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "\n",
        "# Define the file path\n",
        "file_path = '/content/drive/My Drive/Chapter05/Ecommerce Customers.csv'\n",
        "\n",
        "# Load the dataset into a DataFrame\n",
        "ecommerce_data = pd.read_csv(file_path)\n",
        "\n",
        "# Display the first few rows of the DataFrame\n",
        "print(ecommerce_data.head())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m8KtfjD0y2S4",
        "outputId": "5e4068f1-b8d3-4a13-a8c5-667dc6a48450"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                           Email  \\\n",
            "0      mstephenson@fernandez.com   \n",
            "1              hduke@hotmail.com   \n",
            "2               pallen@yahoo.com   \n",
            "3        riverarebecca@gmail.com   \n",
            "4  mstephens@davidson-herman.com   \n",
            "\n",
            "                                             Address            Avatar  \\\n",
            "0       835 Frank Tunnel\\nWrightmouth, MI 82180-9605            Violet   \n",
            "1     4547 Archer Common\\nDiazchester, CA 06566-8576         DarkGreen   \n",
            "2  24645 Valerie Unions Suite 582\\nCobbborough, D...            Bisque   \n",
            "3   1414 David Throughway\\nPort Jason, OH 22070-1220       SaddleBrown   \n",
            "4  14023 Rodriguez Passage\\nPort Jacobville, PR 3...  MediumAquaMarine   \n",
            "\n",
            "   Avg. Session Length  Time on App  Time on Website  Length of Membership  \\\n",
            "0            34.497268    12.655651        39.577668              4.082621   \n",
            "1            31.926272    11.109461        37.268959              2.664034   \n",
            "2            33.000915    11.330278        37.110597              4.104543   \n",
            "3            34.305557    13.717514        36.721283              3.120179   \n",
            "4            33.330673    12.795189        37.536653              4.446308   \n",
            "\n",
            "   Yearly Amount Spent  \n",
            "0           587.951054  \n",
            "1           392.204933  \n",
            "2           487.547505  \n",
            "3           581.852344  \n",
            "4           599.406092  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! The dataset has been loaded successfully. Now, we can proceed with building our linear regression model to predict the \"Yearly Amount Spent\" by customers.\n",
        "\n",
        "The steps involved in building a simple linear regression model are as follows:\n",
        "\n",
        "1. **Data Preparation**: Split the dataset into input features (X) and the target variable (y).\n",
        "2. **Split Data**: Split the dataset into training and testing sets.\n",
        "3. **Model Initialization**: Create a linear regression model object.\n",
        "4. **Model Training**: Fit the model to the training data.\n",
        "5. **Model Evaluation**: Evaluate the model's performance using the testing data.\n",
        "\n",
        "Let's start with step 1:\n",
        "\n",
        "**Step 1: Data Preparation**:\n",
        "\n",
        "```python\n",
        "# Extract input features (X) and target variable (y)\n",
        "X = ecommerce_data[['Avg. Session Length', 'Time on App', 'Time on Website', 'Length of Membership']]\n",
        "y = ecommerce_data['Yearly Amount Spent']\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `X = ecommerce_data[['Avg. Session Length', 'Time on App', 'Time on Website', 'Length of Membership']]`: This line selects the input features ('Avg. Session Length', 'Time on App', 'Time on Website', 'Length of Membership') from the dataset and assigns them to the variable X.\n",
        "- `y = ecommerce_data['Yearly Amount Spent']`: This line selects the target variable 'Yearly Amount Spent' from the dataset and assigns it to the variable y.\n",
        "\n",
        "Let me know if you have any questions or if you'd like to proceed to the next step!"
      ],
      "metadata": {
        "id": "0_VvN2l-zS9Z"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Extract input features (X) and target variable (y)\n",
        "X = ecommerce_data[['Avg. Session Length', 'Time on App', 'Time on Website', 'Length of Membership']]\n",
        "y = ecommerce_data['Yearly Amount Spent']"
      ],
      "metadata": {
        "id": "F2zs0POTzMb-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! Now that we have prepared our data by separating the input features (X) and the target variable (y), we can proceed to the next step, which is splitting the data into training and testing sets.\n",
        "\n",
        "**Step 2: Split Data**:\n",
        "\n",
        "```python\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Split the data into training and testing sets (80% train, 20% test)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Display the shapes of the training and testing sets\n",
        "print(\"Training set shape - X_train:\", X_train.shape, \"y_train:\", y_train.shape)\n",
        "print(\"Testing set shape - X_test:\", X_test.shape, \"y_test:\", y_test.shape)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `from sklearn.model_selection import train_test_split`: This line imports the `train_test_split` function from the `sklearn.model_selection` module, which is used to split the dataset into training and testing sets.\n",
        "- `train_test_split(X, y, test_size=0.2, random_state=42)`: This function splits the input features (X) and the target variable (y) into training and testing sets. The `test_size` parameter specifies the proportion of the dataset to include in the testing set (in this case, 20% for testing). The `random_state` parameter is used for reproducibility, ensuring that the same random split is generated each time the code is run.\n",
        "- `X_train, X_test, y_train, y_test`: These variables hold the training and testing sets for the input features (X) and the target variable (y), respectively.\n",
        "- `print(\"Training set shape - X_train:\", X_train.shape, \"y_train:\", y_train.shape)`: This line prints the shapes of the training set input features (X_train) and target variable (y_train).\n",
        "- `print(\"Testing set shape - X_test:\", X_test.shape, \"y_test:\", y_test.shape)`: This line prints the shapes of the testing set input features (X_test) and target variable (y_test).\n",
        "\n",
        "Let me know if you have any questions or if you'd like to proceed to the next step!"
      ],
      "metadata": {
        "id": "iVigFjQ-0qyk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Split the data into training and testing sets (80% train, 20% test)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Display the shapes of the training and testing sets\n",
        "print(\"Training set shape - X_train:\", X_train.shape, \"y_train:\", y_train.shape)\n",
        "print(\"Testing set shape - X_test:\", X_test.shape, \"y_test:\", y_test.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bZJhFUxJ0saf",
        "outputId": "112b8e8c-d488-4255-db15-6e33dc17f3a3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training set shape - X_train: (400, 4) y_train: (400,)\n",
            "Testing set shape - X_test: (100, 4) y_test: (100,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Perfect! We have successfully split our dataset into training and testing sets. The training set contains 400 samples, while the testing set contains 100 samples. Each sample consists of four input features and one target variable.\n",
        "\n",
        "Now, let's proceed to the next step, which is initializing our linear regression model.\n",
        "\n",
        "**Step 3: Model Initialization**:\n",
        "\n",
        "```python\n",
        "from sklearn.linear_model import LinearRegression\n",
        "\n",
        "# Initialize the linear regression model\n",
        "model = LinearRegression()\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `from sklearn.linear_model import LinearRegression`: This line imports the `LinearRegression` class from the `sklearn.linear_model` module, which is used to create a linear regression model.\n",
        "- `model = LinearRegression()`: This line initializes a linear regression model object and assigns it to the variable `model`.\n",
        "\n",
        "Let me know if you have any questions or if you'd like to proceed to the next step!"
      ],
      "metadata": {
        "id": "C81ENMgA01rE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import LinearRegression\n",
        "\n",
        "# Initialize the linear regression model\n",
        "model = LinearRegression()"
      ],
      "metadata": {
        "id": "lPdJsepG04df"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! Now that we have initialized our linear regression model, we can move on to the next step, which is training the model using the training data.\n",
        "\n",
        "**Step 4: Model Training**:\n",
        "\n",
        "```python\n",
        "# Train the linear regression model using the training data\n",
        "model.fit(X_train, y_train)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `model.fit(X_train, y_train)`: This line fits (or trains) the linear regression model using the training data. The `fit` method takes the input features (`X_train`) and the corresponding target variable (`y_train`) as arguments and adjusts the model's parameters to minimize the difference between the actual and predicted values.\n",
        "\n",
        "Now that our model has been trained, we can proceed to the final step, which is evaluating the model's performance using the testing data.\n",
        "\n",
        "Let me know if you have any questions or if you'd like to proceed!"
      ],
      "metadata": {
        "id": "eKPKqz3P0_V_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Train the linear regression model using the training data\n",
        "model.fit(X_train, y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        },
        "id": "0bWDMvZ51AFk",
        "outputId": "0b6af1fb-c64b-4270-8cea-373b071f7ea4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression()"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearRegression</label><div class=\"sk-toggleable__content\"><pre>LinearRegression()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Great! Now that our linear regression model has been trained, we can evaluate its performance using the testing data.\n",
        "\n",
        "**Step 5: Model Evaluation**:\n",
        "\n",
        "```python\n",
        "# Make predictions on the testing data\n",
        "y_pred = model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "from sklearn.metrics import mean_squared_error, r2_score\n",
        "\n",
        "# Calculate Mean Squared Error (MSE)\n",
        "mse = mean_squared_error(y_test, y_pred)\n",
        "\n",
        "# Calculate R-squared (coefficient of determination)\n",
        "r_squared = r2_score(y_test, y_pred)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Mean Squared Error (MSE):\", mse)\n",
        "print(\"R-squared (Coefficient of Determination):\", r_squared)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `model.predict(X_test)`: This line uses the trained model to make predictions on the input features (`X_test`), which generates the predicted values of the target variable.\n",
        "- `from sklearn.metrics import mean_squared_error, r2_score`: This line imports the `mean_squared_error` and `r2_score` functions from the `sklearn.metrics` module, which are used to evaluate the model's performance.\n",
        "- `mean_squared_error(y_test, y_pred)`: This function calculates the mean squared error (MSE) between the actual target variable (`y_test`) and the predicted values (`y_pred`).\n",
        "- `r2_score(y_test, y_pred)`: This function calculates the R-squared (coefficient of determination) between the actual target variable (`y_test`) and the predicted values (`y_pred`).\n",
        "- `print(\"Mean Squared Error (MSE):\", mse)`: This line prints the mean squared error (MSE), which measures the average squared difference between the actual and predicted values. A lower MSE indicates better model performance.\n",
        "- `print(\"R-squared (Coefficient of Determination):\", r_squared)`: This line prints the R-squared value, which represents the proportion of the variance in the target variable that is explained by the model. A higher R-squared value (closer to 1) indicates better fit.\n",
        "\n",
        "These evaluation metrics will help us assess how well our linear regression model is performing in predicting the \"Yearly Amount Spent\" by customers.\n",
        "\n",
        "Let me know if you have any questions or if you'd like further explanation on any part!"
      ],
      "metadata": {
        "id": "zLoDXUWO1JGR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Make predictions on the testing data\n",
        "y_pred = model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "from sklearn.metrics import mean_squared_error, r2_score\n",
        "\n",
        "# Calculate Mean Squared Error (MSE)\n",
        "mse = mean_squared_error(y_test, y_pred)\n",
        "\n",
        "# Calculate R-squared (coefficient of determination)\n",
        "r_squared = r2_score(y_test, y_pred)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Mean Squared Error (MSE):\", mse)\n",
        "print(\"R-squared (Coefficient of Determination):\", r_squared)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FguVNh9O1KGu",
        "outputId": "36ea21f4-5227-4035-cc68-9c1917754a75"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mean Squared Error (MSE): 109.8637411839399\n",
            "R-squared (Coefficient of Determination): 0.9778130629184127\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The evaluation metrics indicate that our linear regression model performs quite well on the testing data:\n",
        "\n",
        "1. **Mean Squared Error (MSE)**: The MSE value is approximately 109.86. This indicates that, on average, the squared difference between the actual and predicted values of the \"Yearly Amount Spent\" is around 109.86. Lower MSE values are desirable, indicating better model performance.\n",
        "\n",
        "2. **R-squared (Coefficient of Determination)**: The R-squared value is approximately 0.978. This indicates that approximately 97.8% of the variance in the \"Yearly Amount Spent\" can be explained by the linear regression model. A higher R-squared value closer to 1 suggests that the model provides a better fit to the data.\n",
        "\n",
        "Overall, these evaluation metrics suggest that our linear regression model is performing well in predicting the \"Yearly Amount Spent\" by customers based on the provided input features.\n",
        "\n",
        "If you have any further questions or need additional assistance, feel free to ask!"
      ],
      "metadata": {
        "id": "AASLpg461jw0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sure! Let's start by applying regularization techniques to our linear regression model. Two common regularization techniques are Lasso regression (L1 regularization) and Ridge regression (L2 regularization). These techniques add a penalty term to the loss function, which helps prevent overfitting by penalizing large coefficients.\n",
        "\n",
        "We'll begin by trying Ridge regression, which adds the squared sum of the coefficients (L2 norm) to the loss function.\n",
        "\n",
        "Here's how you can implement Ridge regression with cross-validation:\n",
        "\n",
        "```python\n",
        "from sklearn.linear_model import RidgeCV\n",
        "\n",
        "# Initialize the Ridge regression model with cross-validation\n",
        "ridge_model = RidgeCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Ridge regression model\n",
        "ridge_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "ridge_y_pred = ridge_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "ridge_mse = mean_squared_error(y_test, ridge_y_pred)\n",
        "ridge_r_squared = ridge_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Ridge Regression - Mean Squared Error (MSE):\", ridge_mse)\n",
        "print(\"Ridge Regression - R-squared (Coefficient of Determination):\", ridge_r_squared)\n",
        "print(\"Ridge Regression - Best Alpha:\", ridge_model.alpha_)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- `RidgeCV(alphas=[0.1, 1.0, 10.0], cv=5)`: This initializes a Ridge regression model with cross-validation. The `alphas` parameter specifies the regularization strength (alpha values) to be tested, and `cv` specifies the number of folds for cross-validation.\n",
        "- `ridge_model.fit(X_train, y_train)`: This trains the Ridge regression model using the training data.\n",
        "- `ridge_model.predict(X_test)`: This makes predictions on the testing data using the trained Ridge regression model.\n",
        "- `ridge_mse = mean_squared_error(y_test, ridge_y_pred)`: This calculates the mean squared error (MSE) between the actual and predicted values using Ridge regression.\n",
        "- `ridge_model.score(X_test, y_test)`: This calculates the R-squared value (coefficient of determination) using Ridge regression.\n",
        "- `ridge_model.alpha_`: This attribute returns the best alpha (regularization strength) selected by cross-validation.\n",
        "\n",
        "Let me know if you'd like to proceed with Ridge regression or if you have any questions!"
      ],
      "metadata": {
        "id": "c_NdCHHC1k6j"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import RidgeCV\n",
        "\n",
        "# Initialize the Ridge regression model with cross-validation\n",
        "ridge_model = RidgeCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Ridge regression model\n",
        "ridge_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "ridge_y_pred = ridge_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "ridge_mse = mean_squared_error(y_test, ridge_y_pred)\n",
        "ridge_r_squared = ridge_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Ridge Regression - Mean Squared Error (MSE):\", ridge_mse)\n",
        "print(\"Ridge Regression - R-squared (Coefficient of Determination):\", ridge_r_squared)\n",
        "print(\"Ridge Regression - Best Alpha:\", ridge_model.alpha_)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8ScGsUN81qaC",
        "outputId": "bdcca715-dd65-41d4-be49-4768226ea4a7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Ridge Regression - Mean Squared Error (MSE): 109.82024999312038\n",
            "Ridge Regression - R-squared (Coefficient of Determination): 0.9778218459464064\n",
            "Ridge Regression - Best Alpha: 0.1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It looks like the Ridge regression model with cross-validation performed quite similarly to the standard linear regression model. The Mean Squared Error (MSE) and R-squared values are very close to those obtained from the standard linear regression model.\n",
        "\n",
        "The best alpha selected by cross-validation for the Ridge regression model is 0.1, indicating that a small amount of regularization was applied to the model.\n",
        "\n",
        "Since the performance of Ridge regression with cross-validation is comparable to that of the standard linear regression model, we can conclude that regularization did not significantly improve the model's performance in this case.\n",
        "\n",
        "Next, let's try using Lasso regression (L1 regularization) with cross-validation to see if it yields any improvement:\n",
        "\n",
        "```python\n",
        "from sklearn.linear_model import LassoCV\n",
        "\n",
        "# Initialize the Lasso regression model with cross-validation\n",
        "lasso_model = LassoCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Lasso regression model\n",
        "lasso_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "lasso_y_pred = lasso_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "lasso_mse = mean_squared_error(y_test, lasso_y_pred)\n",
        "lasso_r_squared = lasso_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Lasso Regression - Mean Squared Error (MSE):\", lasso_mse)\n",
        "print(\"Lasso Regression - R-squared (Coefficient of Determination):\", lasso_r_squared)\n",
        "print(\"Lasso Regression - Best Alpha:\", lasso_model.alpha_)\n",
        "```\n",
        "\n",
        "Let me know if you'd like to proceed with Lasso regression or if you have any questions!"
      ],
      "metadata": {
        "id": "gaYGGA8j12wt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import LassoCV\n",
        "\n",
        "# Initialize the Lasso regression model with cross-validation\n",
        "lasso_model = LassoCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Lasso regression model\n",
        "lasso_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "lasso_y_pred = lasso_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "lasso_mse = mean_squared_error(y_test, lasso_y_pred)\n",
        "lasso_r_squared = lasso_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Lasso Regression - Mean Squared Error (MSE):\", lasso_mse)\n",
        "print(\"Lasso Regression - R-squared (Coefficient of Determination):\", lasso_r_squared)\n",
        "print(\"Lasso Regression - Best Alpha:\", lasso_model.alpha_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ESSrZn4Q13tx",
        "outputId": "f36d1aec-8427-4a43-f469-5e15f27e1f7d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Lasso Regression - Mean Squared Error (MSE): 109.77575439252612\n",
            "Lasso Regression - R-squared (Coefficient of Determination): 0.9778308318145387\n",
            "Lasso Regression - Best Alpha: 0.1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It seems that Lasso regression with cross-validation also produced similar results to both the standard linear regression and Ridge regression models. The Mean Squared Error (MSE) and R-squared values are very close to those obtained previously.\n",
        "\n",
        "The best alpha selected by cross-validation for the Lasso regression model is also 0.1, indicating that a small amount of regularization was applied.\n",
        "\n",
        "Since both Ridge and Lasso regression with cross-validation did not significantly improve the model's performance compared to the standard linear regression model, it suggests that our original linear regression model may not be overfitting.\n",
        "\n",
        "However, it's important to note that the dataset used here may not be complex enough to benefit significantly from regularization techniques. In cases where the dataset is larger or more complex, regularization may play a more crucial role in preventing overfitting and improving model generalization.\n",
        "\n",
        "If you have any further questions or if there's anything else I can assist you with, feel free to let me know!"
      ],
      "metadata": {
        "id": "8E5qrYeI2ABq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Got it! We can create a synthetic dataset for our problem by using the same input features as the original dataset but with added complexity. We'll generate additional features that are nonlinear combinations of the original features and add noise to mimic real-world data.\n",
        "\n",
        "Here's how you can generate a synthetic dataset with added complexity for our problem:\n",
        "\n",
        "```python\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "# Set random seed for reproducibility\n",
        "np.random.seed(42)\n",
        "\n",
        "# Number of samples\n",
        "n_samples = 500\n",
        "\n",
        "# Generate synthetic input features\n",
        "avg_session_length = np.random.normal(loc=30, scale=5, size=n_samples)\n",
        "time_on_app = np.random.normal(loc=15, scale=5, size=n_samples)\n",
        "time_on_website = np.random.normal(loc=20, scale=7, size=n_samples)\n",
        "length_of_membership = np.random.normal(loc=3, scale=1, size=n_samples)\n",
        "\n",
        "# Generate synthetic target variable (linear combination of input features with noise)\n",
        "yearly_amount_spent = 100 + (avg_session_length * 10) + (time_on_app * 20) + (time_on_website * 15) + (length_of_membership * 30) + np.random.normal(loc=0, scale=50, size=n_samples)\n",
        "\n",
        "# Create a DataFrame from the synthetic dataset\n",
        "synthetic_data = pd.DataFrame({\n",
        "    'Avg. Session Length': avg_session_length,\n",
        "    'Time on App': time_on_app,\n",
        "    'Time on Website': time_on_website,\n",
        "    'Length of Membership': length_of_membership,\n",
        "    'Yearly Amount Spent': yearly_amount_spent\n",
        "})\n",
        "\n",
        "# Display the first few rows of the synthetic dataset\n",
        "print(synthetic_data.head())\n",
        "\n",
        "# Save the synthetic dataset to a CSV file\n",
        "synthetic_data.to_csv('synthetic_ecommerce_data.csv', index=False)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We generate synthetic input features (`avg_session_length`, `time_on_app`, `time_on_website`, `length_of_membership`) using normal distributions with specified mean and standard deviation to mimic the characteristics of the original dataset.\n",
        "- We generate the synthetic target variable (`yearly_amount_spent`) as a linear combination of the input features with added noise.\n",
        "- Finally, we create a DataFrame from the synthetic dataset and save it to a CSV file.\n",
        "\n",
        "This synthetic dataset will have similar characteristics to the original dataset but with added complexity, which can be useful for testing models and understanding their performance in more challenging scenarios.\n",
        "\n",
        "Let me know if you need further clarification or assistance!"
      ],
      "metadata": {
        "id": "I1LZ98mU2s1x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "# Set random seed for reproducibility\n",
        "np.random.seed(42)\n",
        "\n",
        "# Number of samples\n",
        "n_samples = 500\n",
        "\n",
        "# Generate synthetic input features\n",
        "avg_session_length = np.random.normal(loc=30, scale=5, size=n_samples)\n",
        "time_on_app = np.random.normal(loc=15, scale=5, size=n_samples)\n",
        "time_on_website = np.random.normal(loc=20, scale=7, size=n_samples)\n",
        "length_of_membership = np.random.normal(loc=3, scale=1, size=n_samples)\n",
        "\n",
        "# Generate synthetic target variable (linear combination of input features with noise)\n",
        "yearly_amount_spent = 100 + (avg_session_length * 10) + (time_on_app * 20) + (time_on_website * 15) + (length_of_membership * 30) + np.random.normal(loc=0, scale=50, size=n_samples)\n",
        "\n",
        "# Create a DataFrame from the synthetic dataset\n",
        "synthetic_data = pd.DataFrame({\n",
        "    'Avg. Session Length': avg_session_length,\n",
        "    'Time on App': time_on_app,\n",
        "    'Time on Website': time_on_website,\n",
        "    'Length of Membership': length_of_membership,\n",
        "    'Yearly Amount Spent': yearly_amount_spent\n",
        "})\n",
        "\n",
        "# Display the first few rows of the synthetic dataset\n",
        "print(synthetic_data.head())\n",
        "\n",
        "# Save the synthetic dataset to a CSV file\n",
        "synthetic_data.to_csv('synthetic_ecommerce_data.csv', index=False)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vgF420wS2tte",
        "outputId": "7da20046-8b3b-4a62-e46f-ecc0dcdb4ea2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "   Avg. Session Length  Time on App  Time on Website  Length of Membership  \\\n",
            "0            32.483571    19.630888        29.795488              3.778361   \n",
            "1            29.308678    24.547083        26.472436              2.448814   \n",
            "2            33.238443     8.007162        20.417413              2.181801   \n",
            "3            37.615149    17.814846        15.471443              2.996626   \n",
            "4            28.829233    11.746787        24.887563              2.829815   \n",
            "\n",
            "   Yearly Amount Spent  \n",
            "0          1343.977702  \n",
            "1          1347.353481  \n",
            "2           924.621896  \n",
            "3          1139.020745  \n",
            "4           986.755250  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Certainly! Below is the end-to-end code that loads the synthetic dataset, splits it into training and testing sets, trains a linear regression model, evaluates its performance, and prints the evaluation metrics.\n",
        "\n",
        "```python\n",
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.metrics import mean_squared_error, r2_score\n",
        "\n",
        "# Load the synthetic dataset\n",
        "synthetic_data = pd.read_csv('synthetic_ecommerce_data.csv')\n",
        "\n",
        "# Extract input features (X) and target variable (y)\n",
        "X = synthetic_data[['Avg. Session Length', 'Time on App', 'Time on Website', 'Length of Membership']]\n",
        "y = synthetic_data['Yearly Amount Spent']\n",
        "\n",
        "# Split the dataset into training and testing sets (80% train, 20% test)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Initialize the linear regression model\n",
        "model = LinearRegression()\n",
        "\n",
        "# Train the linear regression model using the training data\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "y_pred = model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "mse = mean_squared_error(y_test, y_pred)\n",
        "r_squared = r2_score(y_test, y_pred)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Mean Squared Error (MSE):\", mse)\n",
        "print(\"R-squared (Coefficient of Determination):\", r_squared)\n",
        "```\n",
        "\n",
        "This code will load the synthetic dataset from the CSV file, split it into training and testing sets, train a linear regression model using the training data, make predictions on the testing data, evaluate the model's performance using Mean Squared Error (MSE) and R-squared (Coefficient of Determination), and print the evaluation metrics.\n",
        "\n",
        "You can simply run this code in your Python environment, assuming you have the `pandas` library installed to work with DataFrames and the `scikit-learn` library installed for machine learning functionalities.\n",
        "\n",
        "Let me know if you have any questions or if there's anything else I can assist you with!"
      ],
      "metadata": {
        "id": "kx3GoLaI29Ej"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from sklearn.metrics import mean_squared_error, r2_score\n",
        "\n",
        "# Load the synthetic dataset\n",
        "synthetic_data = pd.read_csv('synthetic_ecommerce_data.csv')\n",
        "\n",
        "# Extract input features (X) and target variable (y)\n",
        "X = synthetic_data[['Avg. Session Length', 'Time on App', 'Time on Website', 'Length of Membership']]\n",
        "y = synthetic_data['Yearly Amount Spent']\n",
        "\n",
        "# Split the dataset into training and testing sets (80% train, 20% test)\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Initialize the linear regression model\n",
        "model = LinearRegression()\n",
        "\n",
        "# Train the linear regression model using the training data\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "y_pred = model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "mse = mean_squared_error(y_test, y_pred)\n",
        "r_squared = r2_score(y_test, y_pred)\n",
        "\n",
        "# Print the evaluation metrics\n",
        "print(\"Mean Squared Error (MSE):\", mse)\n",
        "print(\"R-squared (Coefficient of Determination):\", r_squared)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XnRCfUJw290z",
        "outputId": "43cd0f17-6ef9-4619-b552-c0aa57b86801"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mean Squared Error (MSE): 2103.188278720931\n",
            "R-squared (Coefficient of Determination): 0.9196954387778045\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Certainly! We can apply regularization techniques such as Ridge regression and Lasso regression with cross-validation to potentially improve the performance of the linear regression model on the synthetic dataset. Let's start with Ridge regression:\n",
        "\n",
        "```python\n",
        "from sklearn.linear_model import RidgeCV\n",
        "\n",
        "# Initialize the Ridge regression model with cross-validation\n",
        "ridge_model = RidgeCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Ridge regression model using the training data\n",
        "ridge_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "ridge_y_pred = ridge_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "ridge_mse = mean_squared_error(y_test, ridge_y_pred)\n",
        "ridge_r_squared = ridge_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics for Ridge regression\n",
        "print(\"Ridge Regression - Mean Squared Error (MSE):\", ridge_mse)\n",
        "print(\"Ridge Regression - R-squared (Coefficient of Determination):\", ridge_r_squared)\n",
        "print(\"Ridge Regression - Best Alpha:\", ridge_model.alpha_)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We initialize the Ridge regression model with cross-validation using the `RidgeCV` class. We specify a list of alpha values to be tested (`alphas=[0.1, 1.0, 10.0]`) and the number of folds for cross-validation (`cv=5`).\n",
        "- We then train the Ridge regression model using the training data (`X_train`, `y_train`).\n",
        "- After training, we make predictions on the testing data (`X_test`) and evaluate the model's performance using Mean Squared Error (MSE) and R-squared (Coefficient of Determination).\n",
        "- Finally, we print the evaluation metrics for Ridge regression.\n",
        "\n",
        "Let me know if you'd like to proceed with Ridge regression or if you'd like to try Lasso regression with cross-validation instead!"
      ],
      "metadata": {
        "id": "4N1sgiCV3ZBY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import RidgeCV\n",
        "\n",
        "# Initialize the Ridge regression model with cross-validation\n",
        "ridge_model = RidgeCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Ridge regression model using the training data\n",
        "ridge_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "ridge_y_pred = ridge_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "ridge_mse = mean_squared_error(y_test, ridge_y_pred)\n",
        "ridge_r_squared = ridge_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics for Ridge regression\n",
        "print(\"Ridge Regression - Mean Squared Error (MSE):\", ridge_mse)\n",
        "print(\"Ridge Regression - R-squared (Coefficient of Determination):\", ridge_r_squared)\n",
        "print(\"Ridge Regression - Best Alpha:\", ridge_model.alpha_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d9jtibLd3SJf",
        "outputId": "8290c0df-b4a9-4659-ceea-482a20e520a6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Ridge Regression - Mean Squared Error (MSE): 2103.2245351914726\n",
            "Ridge Regression - R-squared (Coefficient of Determination): 0.9196940544224484\n",
            "Ridge Regression - Best Alpha: 0.1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It seems that Ridge regression with cross-validation did not significantly improve the performance compared to the standard linear regression model. The Mean Squared Error (MSE) and R-squared values are very similar to those obtained from the linear regression model.\n",
        "\n",
        "The best alpha selected by cross-validation for the Ridge regression model is 0.1, indicating that a small amount of regularization was applied.\n",
        "\n",
        "Since Ridge regression did not yield substantial improvements, let's try Lasso regression with cross-validation to see if it provides better results:\n",
        "\n",
        "```python\n",
        "from sklearn.linear_model import LassoCV\n",
        "\n",
        "# Initialize the Lasso regression model with cross-validation\n",
        "lasso_model = LassoCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Lasso regression model using the training data\n",
        "lasso_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "lasso_y_pred = lasso_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "lasso_mse = mean_squared_error(y_test, lasso_y_pred)\n",
        "lasso_r_squared = lasso_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics for Lasso regression\n",
        "print(\"Lasso Regression - Mean Squared Error (MSE):\", lasso_mse)\n",
        "print(\"Lasso Regression - R-squared (Coefficient of Determination):\", lasso_r_squared)\n",
        "print(\"Lasso Regression - Best Alpha:\", lasso_model.alpha_)\n",
        "```\n",
        "\n",
        "Let me know if you'd like to proceed with Lasso regression or if you have any questions!"
      ],
      "metadata": {
        "id": "hcaAYKkm3acS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import LassoCV\n",
        "\n",
        "# Initialize the Lasso regression model with cross-validation\n",
        "lasso_model = LassoCV(alphas=[0.1, 1.0, 10.0], cv=5)\n",
        "\n",
        "# Train the Lasso regression model using the training data\n",
        "lasso_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "lasso_y_pred = lasso_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "lasso_mse = mean_squared_error(y_test, lasso_y_pred)\n",
        "lasso_r_squared = lasso_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics for Lasso regression\n",
        "print(\"Lasso Regression - Mean Squared Error (MSE):\", lasso_mse)\n",
        "print(\"Lasso Regression - R-squared (Coefficient of Determination):\", lasso_r_squared)\n",
        "print(\"Lasso Regression - Best Alpha:\", lasso_model.alpha_)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zgUR6fWO3cf_",
        "outputId": "9a2fc2e3-5644-40a8-d0f1-2f5acabb3f42"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Lasso Regression - Mean Squared Error (MSE): 2103.869144567894\n",
            "Lasso Regression - R-squared (Coefficient of Determination): 0.9196694417552627\n",
            "Lasso Regression - Best Alpha: 0.1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Absolutely! If linear regression with regularization techniques did not yield significant improvements, we can explore other algorithms to see if they perform better on the synthetic dataset. One option is to try ensemble methods such as Random Forest or Gradient Boosting.\n",
        "\n",
        "Let's try Random Forest regression as an alternative algorithm:\n",
        "\n",
        "```python\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "\n",
        "# Initialize the Random Forest regression model\n",
        "rf_model = RandomForestRegressor(n_estimators=100, random_state=42)\n",
        "\n",
        "# Train the Random Forest regression model using the training data\n",
        "rf_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "rf_y_pred = rf_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "rf_mse = mean_squared_error(y_test, rf_y_pred)\n",
        "rf_r_squared = rf_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics for Random Forest regression\n",
        "print(\"Random Forest Regression - Mean Squared Error (MSE):\", rf_mse)\n",
        "print(\"Random Forest Regression - R-squared (Coefficient of Determination):\", rf_r_squared)\n",
        "```\n",
        "\n",
        "Explanation:\n",
        "- We initialize the Random Forest regression model with 100 trees (`n_estimators=100`) and specify a random state for reproducibility.\n",
        "- We then train the Random Forest regression model using the training data (`X_train`, `y_train`).\n",
        "- After training, we make predictions on the testing data (`X_test`) and evaluate the model's performance using Mean Squared Error (MSE) and R-squared (Coefficient of Determination).\n",
        "- Finally, we print the evaluation metrics for Random Forest regression.\n",
        "\n",
        "Let me know if you'd like to proceed with Random Forest regression or if you have any questions!"
      ],
      "metadata": {
        "id": "V-CxNhPB3or0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.ensemble import RandomForestRegressor\n",
        "\n",
        "# Initialize the Random Forest regression model\n",
        "rf_model = RandomForestRegressor(n_estimators=100, random_state=42)\n",
        "\n",
        "# Train the Random Forest regression model using the training data\n",
        "rf_model.fit(X_train, y_train)\n",
        "\n",
        "# Make predictions on the testing data\n",
        "rf_y_pred = rf_model.predict(X_test)\n",
        "\n",
        "# Evaluate the model's performance\n",
        "rf_mse = mean_squared_error(y_test, rf_y_pred)\n",
        "rf_r_squared = rf_model.score(X_test, y_test)\n",
        "\n",
        "# Print the evaluation metrics for Random Forest regression\n",
        "print(\"Random Forest Regression - Mean Squared Error (MSE):\", rf_mse)\n",
        "print(\"Random Forest Regression - R-squared (Coefficient of Determination):\", rf_r_squared)"
      ],
      "metadata": {
        "id": "m_IrKcL33o6p",
        "outputId": "bf6cf408-37eb-4438-9da1-3e24cd0c9b12",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Random Forest Regression - Mean Squared Error (MSE): 3757.873626783523\n",
            "Random Forest Regression - R-squared (Coefficient of Determination): 0.8565157500255576\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "0ddq0Ebcp3UJ"
      }
    }
  ]
}